🐛 fix: MiniMax output long content interrupted by non-existent error #4088

sxjeru · 2024-09-23T04:59:24Z

💻 变更类型 | Change Type

🔀 变更说明 | Description of Change

去除了默认的 frequency_penalty 参数。

同时结合上下文窗口等模型信息，优化了默认的 max_tokens 参数。

另外由于 minimax 流式输出在最后会一次性输出所有内容，导致最后一个 chunk 无法被解析为 json，故常在生成完毕时报错。目前增加了处理流程，去除最后一次的 data 流数据，以修复此问题。

~~个人没有条件测试，可以测试后再合并。~~
测试可用。

📝 补充信息 | Additional Information

fix #4054

vercel · 2024-09-23T04:59:28Z

@sxjeru is attempting to deploy a commit to the LobeHub Pro Team on Vercel.

A member of the Team first needs to authorize it.

lobehubbot · 2024-09-23T04:59:37Z

👍 @sxjeru

Thank you for raising your pull request and contributing to our Community
Please make sure you have followed our contributing guidelines. We will review it as soon as possible.
If you encounter any problems, please feel free to connect with us.
非常感谢您提出拉取请求并为我们的社区做出贡献，请确保您已经遵循了我们的贡献指南，我们会尽快审查它。
如果您遇到任何问题，请随时与我们联系。

codecov · 2024-09-23T05:02:29Z

Codecov Report

Attention: Patch coverage is 93.54839% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/libs/agent-runtime/minimax/index.ts	87.50%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4088      +/-   ##
==========================================
+ Coverage   92.45%   92.48%   +0.02%     
==========================================
  Files         482      482              
  Lines       34354    34497     +143     
  Branches     3214     3232      +18     
==========================================
+ Hits        31763    31905     +142     
- Misses       2591     2592       +1

Flag	Coverage Δ
app	`92.48% <93.54%> (+0.02%)`	⬆️
server	`97.36% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

vercel · 2024-09-23T05:04:49Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
lobe-chat-preview	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Sep 25, 2024 3:12pm

LovelyGuYiMeng · 2024-09-23T06:38:11Z

{
  "message": "chat response streaming chunk parse error, please contact your API Provider to fix it.",
  "context": {
    "error": {
      "message": "Cannot read properties of undefined (reading '0')",
      "name": "TypeError"
    },
    "chunk": {
      "raw": "4890\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\"村民们一个接一个地喝下了药剂，他们的病情开始好转，最终全部康复。莉\",\"role\":\"assistant\",\"name\":\"MM智能助理\",\"audio_content\":\"\"}}],\"created\":1727073422,\"model\":\"abab6.5s-chat\",\"object\":\"chat.completion.chunk\",\"usage\":{\"total_tokens\":0,\"total_characters\":0},\"input_sensitive\":false,\"output_sensitive\":false,\"input_sensitive_type\":0,\"output_sensitive_type\":0,\"output_sensitive_int\":0}\n\n"
    }
  }
}

该版本依旧输出长文本中断

lobehubbot · 2024-09-23T06:38:22Z

{
"message": "chat response streaming chunk parse error, please contact your API Provider to fix it.",
"context": {
"error": {
"message": "Cannot read properties of undefined (reading '0')",
"name": "TypeError"
},
"chunk": {
"raw": "4890","choices":[{"index":0,"delta":{"content":"The villagers drank the potion one after another , their conditions began to improve, and eventually they all recovered. "}}],"created":1727073422,"model":"abab6.5s-chat","object":"chat.completion.chunk","usage" :{"total_tokens":0,"total_characters":0},"input_sensitive":false,"output_sensitive":false,"input_sensitive_type":0,"output_sensitive_type":0 ,"output_sensitive_int":0}\n\n"
}
}
}

This version still outputs long text interruptions

LovelyGuYiMeng · 2024-09-23T08:40:22Z

maxtoken要限制在4k和8k吗，明明有更高的上限

lobehubbot · 2024-09-23T08:40:34Z

Should maxtoken be limited to 4k and 8k? There is obviously a higher limit

LovelyGuYiMeng · 2024-09-23T08:44:44Z

当前版本长文本不再报错，但仍然会中断

lobehubbot · 2024-09-23T08:44:57Z

In the current version, long text no longer reports an error, but it will still be interrupted.

arvinxx · 2024-09-23T08:54:09Z

有文档吗

lobehubbot · 2024-09-23T08:54:23Z

Is there any documentation?

LovelyGuYiMeng · 2024-09-23T08:55:19Z

lobehubbot · 2024-09-23T08:55:31Z

sxjeru · 2024-09-23T08:55:35Z

max_tokens 可以在 lobechat 手动调，目前是给了一个较大的默认值。（相较于原本的 256）

请先不要合并，还有问题待修复，参数只是其中一个问题。

lobehubbot · 2024-09-23T08:55:49Z

max_tokens can be adjusted manually in lobechat, and currently a larger default value is given. (compared to the original 256)

Please don't merge yet, there are still problems to be fixed, and parameters are just one of them.

arvinxx · 2024-09-23T08:58:35Z

@sxjeru 建议按 Minimax 的文档来给max_token 值吧。用户不一定会意识到还需要手动改下才能让 6.5s 输出到200k+ 。

但确定这个值是正确的吗？6.5s 确定能输出 200k 的output？我个人有点不太相信

lobehubbot · 2024-09-23T08:58:48Z

@sxjeru suggests setting the max_token value according to the Minimax documentation. Users may not necessarily realize that they need to manually change it to output 200k+ in 6.5s.

But are you sure this value is correct? Are you sure it can output 200k output in 6.5s? I personally don't believe it

sxjeru · 2024-09-23T09:03:48Z

文档是这样写的。
https://platform.minimaxi.com/document/ChatCompletion%20v2?key=66701d281d57f38758d581d0

这边比较混淆的是 minimax 参数里的 max_tokens 相当于 openai 的 Max output tokens，而不是上下文窗口大小，所以我才会比较谨慎地设定。

lobehubbot · 2024-09-23T09:04:01Z

The documentation says this.
https://platform.minimaxi.com/document/ChatCompletion%20v2?key=66701d281d57f38758d581d0

What is confusing here is that max_tokens in the minimax parameter is equivalent to openai's Max output tokens, so I will set it more carefully.

arvinxx · 2024-09-23T09:08:44Z

建议最好测下 output 输出是否能到 20k… 如果能到 20k 肯定就没问题

sxjeru · 2024-09-23T14:31:17Z

应该修好了。minimax 的流式输出有点意思，最后会一次性再输出一次所有内容。

lobehubbot · 2024-09-23T14:31:32Z

It should be fixed.

src/libs/agent-runtime/utils/streams/minimax.ts

arvinxx · 2024-09-24T17:01:10Z

@sxjeru ci 挂了

lobehubbot · 2024-09-24T17:01:22Z

@sxjeru ci died

sxjeru · 2024-09-25T04:59:42Z

已修复。

lobehubbot · 2024-09-25T04:59:55Z

Fixed.

arvinxx · 2024-09-25T14:44:27Z

@sxjeru 构建挂了

arvinxx · 2024-09-25T15:23:29Z

@sxjeru @LovelyGuYiMeng 说还有截断的问题，另外开一个 PR 修吧

lobehubbot · 2024-09-25T15:23:44Z

@sxjeru @LovelyGuYiMeng said there is still a truncation problem, please open another PR to fix it

lobehubbot · 2024-09-25T15:23:53Z

❤️ Great PR @sxjeru ❤️

The growth of project is inseparable from user feedback and contribution, thanks for your contribution! If you are interesting with the lobehub developer community, please join our discord and then dm @arvinxx or @canisminor1990. They will invite you to our private developer channel. We are talking about the lobe-chat development or sharing ai newsletter around the world.
项目的成长离不开用户反馈和贡献，感谢您的贡献! 如果您对 LobeHub 开发者社区感兴趣，请加入我们的 discord，然后私信 @arvinxx 或 @canisminor1990。他们会邀请您加入我们的私密开发者频道。我们将会讨论关于 Lobe Chat 的开发，分享和讨论全球范围内的 AI 消息。

LovelyGuYiMeng · 2024-09-25T15:25:02Z

最新构建情况

sxjeru · 2024-09-25T15:30:24Z

用客户端请求就是正常的，不清楚是超时还是什么原因。

lobehubbot · 2024-09-25T15:30:35Z

It's normal to use the client to request. I don't know if it's a timeout or what the reason is.

### [Version 1.19.33](v1.19.32...v1.19.33) <sup>Released on **2024-09-25**</sup> #### 🐛 Bug Fixes - **misc**: MiniMax output long content interrupted by non-existent error. #### 💄 Styles - **misc**: Update google provider model info. <br/> <details> <summary><kbd>Improvements and Fixes</kbd></summary> #### What's fixed * **misc**: MiniMax output long content interrupted by non-existent error, closes [#4088](#4088) ([4f6e20d](4f6e20d)) #### Styles * **misc**: Update google provider model info, closes [#4129](#4129) ([b1442b9](b1442b9)) </details> <div align="right"> [![](https://img.shields.io/badge/-BACK_TO_TOP-151515?style=flat-square)](#readme-top) </div>

lobehubbot · 2024-09-25T15:31:58Z

🎉 This PR is included in version 1.19.33 🎉

The release is available on:

Your semantic-release bot 📦🚀

### [Version 1.62.14](v1.62.13...v1.62.14) <sup>Released on **2024-09-25**</sup> #### 🐛 Bug Fixes - **misc**: MiniMax output long content interrupted by non-existent error. #### 💄 Styles - **misc**: Add function call for `taichu_llm`, update google provider model info. <br/> <details> <summary><kbd>Improvements and Fixes</kbd></summary> #### What's fixed * **misc**: MiniMax output long content interrupted by non-existent error, closes [lobehub#4088](https://github.com/bentwnghk/lobe-chat/issues/4088) ([4f6e20d](4f6e20d)) #### Styles * **misc**: Add function call for `taichu_llm`, closes [lobehub#4119](https://github.com/bentwnghk/lobe-chat/issues/4119) ([8f629d8](8f629d8)) * **misc**: Update google provider model info, closes [lobehub#4129](https://github.com/bentwnghk/lobe-chat/issues/4129) ([b1442b9](b1442b9)) </details> <div align="right"> [![](https://img.shields.io/badge/-BACK_TO_TOP-151515?style=flat-square)](#readme-top) </div>

Update index.ts

da6bbca

dosubot bot added the size:S This PR changes 10-29 lines, ignoring generated files. label Sep 23, 2024

Update index.test.ts

8afc41c

Update index.ts

096ba5c

dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:S This PR changes 10-29 lines, ignoring generated files. labels Sep 23, 2024

Merge branch 'main' into patch-1

38a5edb

vercel bot deployed to Preview – lobe-chat-preview September 23, 2024 08:41 View deployment

sxjeru changed the title ~~🐛 fix: Avoid introducing frequency_penalty param for MiniMax~~ 🐛 fix: MiniMax output long content interrupted by non-existent error Sep 23, 2024

fix

5b2f749

arvinxx reviewed Sep 23, 2024

View reviewed changes

src/libs/agent-runtime/utils/streams/minimax.ts Show resolved Hide resolved

vercel bot deployed to Preview – lobe-chat-preview September 23, 2024 15:56 View deployment

sxjeru added 4 commits September 24, 2024 09:47

Update minimax.ts

f78e53e

Create minimax.test.ts

0dcb95e

Merge branch 'main' into patch-1

0edebc0

Update minimax.ts

5c13577

Update minimax.ts

83264d0

vercel bot had a problem deploying to Preview – lobe-chat-preview September 25, 2024 14:41 Failure

Update minimax.ts

dd07c2f

vercel bot deployed to Preview – lobe-chat-preview September 25, 2024 15:12 View deployment

arvinxx merged commit 4f6e20d into lobehub:main Sep 25, 2024
6 of 8 checks passed

sxjeru deleted the patch-1 branch September 25, 2024 15:27

lobehubbot added the released label Sep 25, 2024

🐛 fix: MiniMax output long content interrupted by non-existent error #4088

🐛 fix: MiniMax output long content interrupted by non-existent error #4088

Conversation

sxjeru commented Sep 23, 2024 • edited Loading

💻 变更类型 | Change Type

🔀 变更说明 | Description of Change

📝 补充信息 | Additional Information

vercel bot commented Sep 23, 2024

lobehubbot commented Sep 23, 2024

codecov bot commented Sep 23, 2024 • edited Loading

Codecov Report

vercel bot commented Sep 23, 2024 • edited Loading

LovelyGuYiMeng commented Sep 23, 2024 • edited by arvinxx Loading

lobehubbot commented Sep 23, 2024

LovelyGuYiMeng commented Sep 23, 2024

lobehubbot commented Sep 23, 2024

LovelyGuYiMeng commented Sep 23, 2024

lobehubbot commented Sep 23, 2024

arvinxx commented Sep 23, 2024

lobehubbot commented Sep 23, 2024

LovelyGuYiMeng commented Sep 23, 2024

lobehubbot commented Sep 23, 2024

sxjeru commented Sep 23, 2024

lobehubbot commented Sep 23, 2024

arvinxx commented Sep 23, 2024

lobehubbot commented Sep 23, 2024

sxjeru commented Sep 23, 2024 • edited Loading

lobehubbot commented Sep 23, 2024

arvinxx commented Sep 23, 2024

sxjeru commented Sep 23, 2024 • edited Loading

lobehubbot commented Sep 23, 2024

arvinxx commented Sep 24, 2024

lobehubbot commented Sep 24, 2024

sxjeru commented Sep 25, 2024

lobehubbot commented Sep 25, 2024

arvinxx commented Sep 25, 2024

arvinxx commented Sep 25, 2024

lobehubbot commented Sep 25, 2024

lobehubbot commented Sep 25, 2024

LovelyGuYiMeng commented Sep 25, 2024

sxjeru commented Sep 25, 2024

lobehubbot commented Sep 25, 2024

lobehubbot commented Sep 25, 2024

sxjeru commented Sep 23, 2024 •

edited

Loading

codecov bot commented Sep 23, 2024 •

edited

Loading

vercel bot commented Sep 23, 2024 •

edited

Loading

LovelyGuYiMeng commented Sep 23, 2024 •

edited by arvinxx

Loading

sxjeru commented Sep 23, 2024 •

edited

Loading

sxjeru commented Sep 23, 2024 •

edited

Loading